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ON  COMMONALITIES  IN  SIGNAL  DESIGN  FOR 
NON-GAUSSIAN  CHANNELS 


1.  INTRODUCTION 


Signal  design  is  an  important  aspect  in  the  overall  design  of  a  communication  system.  Ideally, 
the  optimal  signal  set,  subject  to  certain  constraints  such  as  bandwidth  or  energy,  minimizes  either 
the  probability  of  error  (Pg)  or  the  Ney man- Pearson  performance  (NP).  Even  in  the  fortuitous  case 
that  the  channel  is  modeled  as  an  additive  Gaussian  noise  channel  (possibly  colored),  there  are  few 
analytic  results.  Moreover,  if  the  noise  happens  to  be  non-Gaussian  in  nature,  the  design  problem 
as  described  above  becomes  for  alL  practical  purposes  analytically  intractable. 

To  address  this  in  this  report  we  apply  and  extend  results  from  Large  Deviation  Theory  (LDT) 
to  the  problem  of  signal  design  for  non-Gaussian  channels.  Originally,  Johnson  and  Orsak  consid¬ 
ered  this  approach  in  Ref.  1  where  they  focused  on  the  design  of  signal  waveforms  which  were 
asymptotically  optimal  with  respect  to  the  Neyman-Pearson  criterion.  We  seek  to  generalize  these 
results  to  determine  signal  sets  that  are  simultaneously  asymptotically  optimal  with  respect  to  the 
minimum  probability  of  error  (Pg),  the  mini-max,  and  the  Neyman-Pearson  criteria. 

One  of  the  main  issues  addressed  in  this  research  is  best  summarized  by  the  following  ques¬ 
tion:  Are  signal  sets  operating  in  non-Gaussian  environments  that  are  optimal  with  respect  to  the 
Neyman-Pearson  criterion  also  optimal  with  respect  to  the  minimum  Pg  and  the  mini-max  criteria? 
Through  this  work  we  are  able  to  conclusively  answer  “yes,”  provided  that  the  length  of  the  signal 
vector  grows  without  bound. 

In  LDT,  the  min  Pg  and  mini-max  criteria  are  associated  with  the  Chernoff  Distance,  and  the 
NP  criterion  with  the  Kullback-Leibler  distance.  We  are  able  to  establish  that  a  unique  optimal 
signal  (if  exists)  maximizes  both  of  these  distances.  Significantly,  we  show  that  this  maximality 
extends  over  the  whole  class  of  Ali-Silvey  distances. 

We  show  within  this  report  that  if  the  background  noise  is  accurately  modeled  as  a  discrete-time 
generalized  Gaussian  random  process,  then  there  are  only  two  optimal  signal  sets  with  respect  to 
all  of  the  above  optimality  criteria.  If  the  tail  of  the  noise  distribution  diminishes  faster  than  that 
of  the  Gaussian,  then  the  optimal  signal  waveform  subject  to  an  energy  constraint  is  an  impulse, 
that  is,  all  of  the  energy  is  contained  in  a  single  sample  of  the  signal  waveform.  Conversely,  if  the 
tail  diminishes  slower  than  that  of  the  Gaussian,  then  the  optimal  signal  has  constant  amplitude 
over  the  waveform.  Only  in  the  case  of  additive  Gaussian  noise  is  a  time-varying  signal  (except 
for  a  purely  impulsive  signal)  potentially^  optimum.  So,  as  a  by-product,  this  work  implies  that 
sinusoidal  waveforms  can  only  be  optimal  for  the  purely  Gaussian  channel. 
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In  addition  to  determining  the  optimal  signal  waveform,  we  have  been  able  to  analytically 
compare  the  relative  performance  of  these  designs  with  respect  to  the  three  optimality  criteria  of 
interest.  Results  will  show  that  for  “small”  signal  energies,  the  error  exponents  associated  with 
the  minimum  Pg  and  mini-max  criteria  are  one  fourth  of  the  error  exponent  of  the  miss  probability 
(Pm)  in  the  Ney man- Pearson  criterion  for  all  non-Gaussian  channels.  Thus,  when  signal  energies 
are  small,  four  times  as  much  energy  or  four  times  as  much  data  are  required  to  achieve  the  same 
error  exponent  in  the  minimum  Pe /mini-max  performance  as  that  required  for  the  NP  performance. 

Conversely,  for  “large”  signal  energies,  we  have  shown  that  the  error  exponents  for  the  minimum 
Pe  and  mini-max  criteria  are  no  more  than  one  half  the  error  exponent  for  P\f  under  Neyman- 
Pearson  criterion.  This  is  to  be  expected  since  the  Neyman- Pearson  detector  need  only  minimize 
Pm  while  the  minimum  Pg  detector  must  simultaneously  minimize  Pp  (false  alarm  rate)  and  Pm 
and  therefore  can  commit  no  more  than  one  half  of  the  computation  capability  of  the  likelihood 
ratio  test  to  either  of  the  two  error  probabilities. 

Even  stronger  results  are  obtained  for  the  case  of  large  signal  energies  when  the  background 
noise  is  assumed  to  be  from  the  Generalized  Gaussian  family  with  decay  rate  r,  i.e.,  when  the 
noise  density  is  modeled  as  p,,{x)  =  K\  exp{— A'2|a:|’').  If  r  >  1,  then  we  have  shown  that  the  error 
exponent  of  the  minimum  Pg  or  mini-max  performance  is  1/2”  of  the  error  exponent  of  Pm  under 
Neyman- Pearson  constraints.  However,  if  r  <  1,  then  the  minimum  Pe/mini-max  error  exponent 
is  precisely  one  half  of  the  error  exponent  of  Pm  under  the  Neyman- Pearson  criterion.  Therefore, 
as  in  the  small  energy  case  for  r  >  1,  to  equate  the  error  exponents,  one  must  utilize  precisely 
four  times  as  much  energy  in  the  minimum  Pg/mini-max  detection  scheme  as  that  used  in  the  NP 
scheme.  However,  if  r  <  1,  then  one  is  required  to  utilize  1/2^/”  times  more  energy  under  minimum 
Pe/mini-max  consideration  as  that  used  in  NP  considerations. 

To  support  this  theory,  we  have  included  Monte  Carlo  simulations.  These  results  show  that  the 
asymptotic  results  hold  with  striking  precision  even  when  in  decidedly  non-asymptotic  regimes. 

2.  PREVIOUS  WORK 

As  described  in  the  introduction,  Johnson  and  Orsak  [1]  were  apparently  the  first  to  use  Large 
Deviation^  approaches  to  design  signal  waveforms  for  the  non-Gaussian  channel. 

The  results  in  this  report  were  based  upon  a  generalization  of  Stein’s  lemma  first  offered  by 
Kullback  [3].  It  was  shown  that  under  Neyman-Pearson  optimality  criterion,  the  error  exponent 
of  the  miss  probability  is  asymptotically  equivalent  to  the  average  Kullback-Leibler  distance  (also 
known  as  the  divergence)  between  the  probability  measures  corresponding  to  the  two  hypotheses. 
From  this,  the  “optimal”  signal  waveform  in  an  additive  non-Gaussian  channel  was  determined  by 
maximizing  the  Kullback-Leibler  distance  subject  to  an  energy  constraint  on  the  signal  waveform. 

It  should  be  pointed  out  that  others  have  also  considered  maximizing  the  divergence  (or  other 
specific  statistical  distance  measures)  [4,  5,  6]  between  hypotheses  as  a  means  of  designing  “good” 


^Large  Deviation  Theory  (LDT)  is  used  to  estimate  the  probabilities  of  rare  events  [11].  For  a  binary  detection 
problem,  we  are  in  the  regime  of  large  deviation  when  the  separation  between  the  probabilities  of  the  two  hypotheses 
is  sufficiently  large  [2]. 
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signal  waveforms.  Grettenberg  [7]  first  proposed  the  maximum  divergence  criterion  for  the  Gaussian 
channel  based  upon  a  duality  result  originating  with  the  work  by  Bradt  and  Karlin  [8]  where  it  was 
shown  that  the  maximum  divergence  criterion  rendered  the  minimum  probability  of  error  signal 
waveform  for  some  a  priori  probabilities  on  the  hypotheses.  Based  upon  this  result,  Grettenberg  was 
able  to  establish  that  the  Simplex  Conjecture  must  hold  for  some  set  of  input  a  priori  probabilities. 

Unfortunately,  as  pointed  out  by  Kailath  [9],  there  is  no  guarantee  that  the  true  a  priori 
probabilities  will  match  those  required  by  this  duality  principle.  In  addition,  in  this  work  Kailath 
offered  an  alternate  statistical  distance  measure  known  as  the  Bhattacharyya  distance  as  a  means 
of  determining  the  optimal  energy  allocation  in  a  Gaussian  environment.  Empirical  results  seemed 
to  suggest  that  the  Bhattacharyya  distance  offered  solutions  that  were  more  consistent  with  those 
derived  by  considering  the  probability  of  error  as  an  optimality  criterion.  Nevertheless,  as  in  the 
case  of  the  maximum  divergence  criterion,  the  maximum  Bhattacharyya  distance  waveforms  were 
not  guaranteed  to  be  optimum  for  the  true  a  priori  probabilities. 

In  this  work,  we  generalize  the  results  offered  in  Ref.  1  to  consider  not  only  the  NP  criterion, 
but  also  the  minimum  Pe  and  mini-max  criterion  for  the  non-Gaussian  environment.  The  signal 
waveforms  obtained  from  this  analysis  will  asymptotically  minimize  the  desired  performance  for 
every  set  of  a  priori  probabilities  and  therefore  will  not  suffer  from  the  same  kinds  of  theoretical 
limitations  as  those  in  Refs.  7  and  9. 


3.  RELATING  PERFORMANCE  TO  CERTAIN  STATISTICAL  DISTANCE 
MEASURES 

Consider  the  following  binary  detection  problem  where  an  N-dimensional  vector  is  transmitted 
through  an  additive  iid  noise  channel: 

Hq  :  Xi  =  Tii 

U  .  V  i  '' 

// 1  .  A I  —  Tlj  "T  ,  Tlj  Pri  ‘ 

Absolute  signal  location  does  not  determine  performance  when  the  noise  density  is  symmetric. 
As  such,  without  loss  of  generality,  we  have  considered  an  “on-off”  signaling  scheme^  where  the 
observation  and  the  signal  vectors  of  interest  will  be  denoted  to  as  and  respectively.  We 
will  assume  throughout  that  the  density  function  of  the  noise  is  symmetric  and  monotonically 
decreasing. 

The  optimal  detector  computes  the  log-likelihood-ratio  test  (LLRT)  based  on  the  aggregate  of 
N  samples  and  compares  the  output  to  threshold  7: 


1  at  /  ^  Hi 

r/vN\  1  V^i _ Pni^i  Si)  ^ 


^Consider  for  example  sending  binary  symbols  W  =  {0,1}  in  a  quadrature  phase  modulation  system.  Symbol 
W  =  i  selects  a  set  of  signal  samples  s,,i  =  1, N  and  generates  a  phase  signal  s(<)  where  sfiT,'  =  s,  and  T,  is  the 
sampling  period.  Symbol  W  =  0  generates  s(t)  =  0  for  the  symbol  interval.  Two  passband  waveforms  are  generated 
for  each  symbol  interval:  fq(t)  =  sin(wc<-t-s(<))  and  /,(<)  =  cos{(A/ct+s{t)).  At  the  receiver,  the  quadrature-modulated 
signals  are  demodulated,  and  combined  to  recover  a  noisy  version  of  the  sequence  i,  =  s,  n,. 
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where  the  threshold  is  chosen  to  optimize  some  performance  measure.  The  false  alarm  and  miss 
probabilities  that  arise  from  the  N-dimensional  hypothesis  testing  problem  are  defined  as  qa/  = 
Pr{sa2/  Hi\Ho}  and  (3^  =  PT{say  Ho\Hi},  respectively  [lOj. 

In  this  report,  we  seek  to  determine  the  signal  waveform  that  simultaneously  optimizes  each 
of  the  following  three  criteria: 

1  (Neyman- Pearson)  minimize  /3n  such  that  ai\i  <  a. 

2  (minimum  Pg)  minimize  where  t,  =  Pr[//,]. 

3  (mini-max)  minimize  maximum  {0^/,/?^} 

For  the  general  non-Gaussian  channel,  the  optimal  signed  waveform  under  any  of  the  above 
optimality  criteria  is  analytically  intractable.  However,  if  we  allow  the  length  of  the  signal  vector  to 
grow  without  bound,  we  may  readily  relate  the  above  performance  criteria  to  information  theoretic 
quantities  that  are  more  amenable  to  analysis.  This  is  accomplished  through  the  application  of 
results  from  LDT  to  the  detection  problem. 

In  the  case  of  the  Neyman-Pearson  criterion,  we  have  via  a  generalized  version  of  Stein’s  lemma 
[2,  11]  the  following  asymptotic  result; 


Theorem  1  Let  satisfy  as  <  a  where  a  >  0. 

lim  ^\ogm\n0N  =  - 

N-*oo  iV 


Then 


lim  — 
iV-00  N 


N 


Y^dKliSi) 


where  is  the  KuUback-Leibler  distance  (or  divergence)  between  Pn{x)  and  p„(a:  -  s,), 


(3) 

i.e., 


This  result  demonstrates  that  with  respect  to  the  NP  criterion,  the  asymptotic  error  exponent  is 
determined  by  the  average  divergence  across  the  data  vector.  As  such,  the  asymptotically  optimal 
signal  waveform  must  maximize  this  average  divergence.  It  was  this  result  that  was  used  in  Ref.  1 
to  design  signal  waveforms  that  are  optimal  for  applications  where  the  Neyman-Pearson  criterion 
is  appropriate,  e.g.,  radar  applications. 

However,  in  most  communication  applications,  one  prefers  to  use  either  the  minimum  Pe  or 
mini-max  criteria.  We  may  obtain  an  analogous  asymptotic  result  by  offering  the  following  gen¬ 
eralization  of  Sanov’s  theorem  (or  sometimes  referred  to  as  Chernoff’s  theorem)[2,  11].  The  proof 
requires  establishing  asymptotically  tight  upper  bound  and  lower  bound  on  the  error  exponent 
that  asymptotically  converge  to  the  Chemoff  bound.  The  proof,  as  shown  in  Appendix  A,  extends 
standard  versions  based  on  i.i.d.  random  variables  to  the  current  problem  where  individual  samples 
Xi  of  the  vector  are  independently  distributed  according  to  known  translations  of  the  noise 
density,  i.e  A,  ~  Pns,- 
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Theorem  2  Let  =  TToayv  +  Then 

Jim  -^logminPj^  =  Jim  logmin  max  {QAr,/?Ar} 

iV— .00  IS  N— .00  TV 

N 


=  ~  /Pn(^)Pn  ^(X-S,)dx} 

1=1 


1 


N 


1=1 


where  the  so-called  Chernoff  distance  dcis)  is  given  by  maxA{-  log/ p^{x)p\  ^{x  -  s)dx} 


(4) 

(5) 

(6) 


As  opposed  to  NP  considerations,  in  this  case,  the  asymptotic  error  exponent  for  both  the 
minimum  and  the  mini-max  detectors  is  determined  by  the  average  Chernoff  distance  across 
the  data  vector.  Thus,  under  these  optimality  criteria,  the  asymptotically  optimal  signal  waveform 
must  maximize  the  average  Chernoff  information. 

Summarizing,  under  the  consideration  that  the  length  of  the  signal  vector  grows  without  bound, 
the  NP  performance  is  determined  by  the  statistical  distance  measure  dici,  whereas  the  minimum 
Pg /mini-max  performance  is  determined  by  the  statistical  distance  measure  dc-  This  observation 
clearly  begs  the  question:  How  aio  dtcL  and  dc  related  in  the  non-Gaussian  environment?  To 
address  this,  in  the  following  section  we  consider  two  signal  regimes,  one  being  the  case  where  the 
admissible  signal  energy  is  small  and  the  other  being  the  case  where  the  admissible  energy  is  large. 

4.  RELATION  BETWEEN  THE  CHERNOFF  INFORMATION  AND  THE 
DIVERGENCE 

The  Chernoff  Information  and  Kullback-Leibler  distances  are  computed  directly  (see  Appendix 
B)  for  a  selection  of  pdfs  as  shown  in  Table  1.  One  can  observe  that  for  the  non-Gaussian  noise 
models  considered,  there  is  very  KAle  functional  similarity  between  dc  and  d^t-  However,  we 


Table  1  —  Chernoff  Information  and  Kullback-Leibler  Distance 


Note:  Ek  is  the  complete  elliptical  integral  of  the  first  (real  argument)  and  second  (imaginary  argument)  kind.  Ki/i  is  the 
modified  Bessel  function  of  the  second  kind  of  fractional  order  1/4. 
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will  be  able  to  analytically  demonstrate  that  there  are  quite  strong  commonalities  for  these  two 
distance  measures  when  one  considers  both  small  and  large  signal  displacements  s. 

Before  proceeding,  we  wish  to  make  an  interesting  obsc  ‘ion  regarding  the  structural  rela¬ 
tionship  between  dc  and  di^i  and  establish  conditions  foi  .uc  reduction  of  dc  to  the  negative 
exponent  of  the  Bhattacharyya  distance. 

4.1.  Bhattacharyya  Distance,  Kullback-Leibler  Distance  and  Chernoff  Infor¬ 
mation 

Let  dc{^)  =  -  s)dx  then 

Fact  1  For  a  fixed  value  of  s,  the  derivative  of  the  Chernoff  Information  ot  A  =  0, 1  is  equivalent 
to  the  Kullback-Leibler  distance. 


Proof: 


f  +dKL{p{x  -  s),i^x))  @A  =  0 

1  -dKLip{x),p{x  -  s))  ®  A  =  1 

For  symmetric  pdfs,  dKL{p{x),p{x  -  s))  =  dfcUpix  -  s),p{x)).D 


(7) 


Fact  2  For  symmetric  densities,  dc{X)  is  both  symmetric  and  concave  in  A. 


Proof: 

m'tcW 

which  is  negative  by  Schwarz  integral  inequality.D 

Fact  1  shows  that  dci^^)  initially  increases  and  eventually  decreases  with  A  at  the  same  rate. 
Combined  with  Fact  2  it  establishes  that  there  must  be  a  maximum  at  the  center  of  symmetry 
A  =  1.  Figure  1  illustrates  this. 

1  /2  1/2 

Since  the  Bhattacharyya  distance  between  po(a:)  and  pi(x)  is  defined  as  dfi  =  f  Pq'  (x)p/  (x)dx, 
we  have  established  that 


[/p^(x)p‘-^(x-3)log  j 


rOil]  —  J  P^(X)P*  ■'(X-3)c(x  J  p^(x)p'“^(x-s)  log^ 
if  p^(x)p*“^(x— s)(ix] 


Fact  3  For  the  translation  problem  of  distinguishing  between  p{x)  and  p(x  —  s),  the  Chernoff  In¬ 
formation  is  equivalent  to  the  negative  of  the  exponent  of  the  Bhattacharyya  distance  for  symmetric 
densities. 

We  use  A  =  i  and  use  dc  =  -  log  dfl  =  -  log  /  yjpn{x)pn{x  ~  s)dx  as  the  Chernoff  Information 
throughout  this  report. 

Figure  1  also  suggests  a  correlation  between  the  slopes  at  end  points  and  the  maximum 

value  dc-  Namely; 
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Fig.  1  —  Chernoff  Infoimatioa  as  a  function  of 
\  for  symmetric  noise  density  functions 


Conjecture  3  It  is  plausible  that  for  some  symmetrical  density  functions,  maximizing  the  Chernoff 
Information  also  maximizes  the  Kullback-Leibler  distances. 


This  observation  is  certainly  true  for  the  set  of  general  exponential  densities  evaluated  in  this 
report,  as  shown  in  the  analyses  of  small-  and  large-signal  behaviors  of  the  distances  in  Sections 
4.3.  and  4.4.  We  first  offer  a  more  general  theorem  using  the  concept  of  Asymptotic  Most  Favorable 
Statistics  advanced  by  Orsak  and  Paris  in  Ref.  6. 


4.2.  Asymptotically  Most  Favorable  Signal 

Definition  1  Signal  s^  is  called  asymptotically  most  favorable  (AMF)  if  and  only  if  for  any  other 
signal  s‘'^ 


+  7ri/3(5^) 
n'Aoo  TToa{s^)  4-  TTi/Jfs^) 


lim 


>  1  for  arbitrary  (tto.tti) 


(9) 


where  a  and  (3  are  the  false  alarm  and  miss  probabilities  defined  for  the  binary  hypothesis  problem, 
and  ttq.tti  the  a  priori  probabilities. 


For  optimal  detectors,  each  of  the  terms  of  the  fraction  is  the  minimum  probability  of  error 
min  Peis)  as  a  function  of  the  signal  s. 
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Lemma  4  Any  real  convex  function  C(x)  of  real  argument  x  may  be  expressed  in  the  following 

form 

'DO 

C(j:)  =  (/)(j-)  +  ^  1(1  -  TT,.,)  -  7r,,,x|  (10) 

i=i 

where  <f>  is  some  lit  ar  function  and  tti  ,  €  (0, 1),  «  =  an  real. 


Proof:  Let  f{x)  be  l(^  “  ^i,i)  ~  Por  i  =  1,  f{x)  is  a  convex  function  with  one  degree 

of  freedom  controlled  by  which  determines  the  location  of  the  breakpoint  and  the  slopes.  For 
each  additional  value  of  t,  the  sum  continues  to  be  a  convex  function  with  an  additional  degree  of 
freedom.  In  the  limit,  f{x)  constructs  a  convex  function  completely  specified  by  the  infinite  set  of 
{jTj,}.  The  linear  function  0{)  specifies  the  location  of  the  minimum  of  the  convex  function.  □ 

Theorem  5  The  asymptotically  most  favorable  signals^  asymptotically  maximizes  any  f-dix'ergence 
)■>  Pii^»  ))■  Conversely,  any  signals^  that  asymptotically  maximizes  any  f-divergence  is  the 
asymptotically  most  favorable  signal. 


Proof:  For  any  f-divergence  of  the  Ali-Silvey  class  (4) 


d(po,Pi)  = 


(11) 


where  A  is  a  real,  increasing  function,  and  C  is  a  convex  function  over  [0,oo).  By  Lemma  4 


<((w,p.)  =  MC»«>(^)  +  -  IT.,.)  -  <ri.^l) 


dpi, 


(12) 


i=l 


Using  the  identity  min(a,6)  =  ^|a  -(-  6|  ^|a  -  6(,  with  a  =  kq  =  1  -  tti  and  b  =  it 

can  be  shown  that  each  of  the  terms  fpol(l  -  7ri,i)  -  Jri.,^1  is  precisely  (1  -  2min/\.,,),  where 
min  Pe  =  TToQ  +  tri/3,  and  zq  =  1  -  trj . 


Therefore, 


lim 

N—*oo 


d{po{s^),Pi{s^)) 

d(Po(s^),Pi(s^)) 


lim 

N-.00 


( 1  -  2  min  P.,(s^ ))] 
(1  -  2  min  F,.(.s^))] 


(13) 


The  AFM  condition  of  Theorem  5  is  satisfied  for  s'^  if 


lim 

N~-*oo 


min  Pf_,(.s^) 
min  Pr.ifsC') 


>  1 


(M) 


for  each  set  of  priors  (7ro,i,7ri,i). 
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Applying  this  condition,  we  have: 

hm  — — =  hm  — -  >  1  for  some  positive  a,  f 

N^oo  d(po(s'^),Pi(s^))  N^oo  <i){l)  +  a  - 


The  converse  may  be  proved  by  reversing  the  steps  above.  □ 


Corollary  6  The  signal  s^  that  maximizes  the  Chemoff  distance  between  {po(s^ )« Pi(^^ )}  also 
maximizes  any  f-divergence  between  these  distributions. 

Proof:  We  have  shown  in  this  report  that  the  signal  s^  that  maximizes  the  Chernoff  distance  is 
optimal  under  the  minPe  criterion.  This  criterion  is  precisely  the  condition  required  Ly  Theorem 
5.  □ 


4.3.  Small  Signal  Behavior 

It  was  shown  [1]  that  for  small  signal  values,  the  divergence  is  locally  proportional  to  an  Cz 
distance  metric  (e.g.  s^)  where  the  multiplicative  constant  is  one  half  of  Fisher’s  information  for 
location.  Mathematically  this  is  best  stated  as 

lim  — 2= —  =  1  (16) 

3—0 

2 

where  I  is  Fisher’s  Information  for  location.  A  similar  result  for  the  Chernoff  distance  can  be 
easily  established. 

Proposition  7  For  diminishingly  small  values  of  $,  dc(s)  is  locally  an  £2  distance  metric  with 
multiplicative  constant  i.e., 

Iin,^  =  l.  (17) 

3—0  SIT 
8 


Proof:  Let  Pnixf=  ^Pn(x)  and  Pnixf=  i^Pn{x)-  Then  dc(s)  =  -\ogJp„(x)ip„^t{x)2dx  and  the 
first  and  second  derivatives  are 

§-dci,)  =  §,  -  log;p„(x)ip„..(a,)-!rfx  =  u 


^dc(s)  - 


33  — r-.-uv-/  —  r  / 

J  Pnlr)2p„  ^,(x)2dx 

5  /  Pn(x)^  Pn-s(x)^  Pn(x)^(Pr,-,(x)~ipn-,(x)+^p„-,(x)~ip„-4x)^)dx 


if  Pn(x)ip„-,(x)^dx] 

if  Pn(x)i  Pn-i(x)~^  Pn-d(x)dT 


+  (18) 


For  symmetric  and  decreasing  densities,  fTooPu{x)dx  =  0  and  p„(x)dx  =  0,  therefore 

,.  j  If  Pn{xfj  \ 

-  hrn  — dc  =  7  /  — 

3-0  ds^  4  J  Pn\x)  4 


(19) 


10 


Chu,  Warke,  and  Orsak 


where  1(5)  is  the  Fisher  Information  of  the  location  s.  dc{s)  may  be  written  in  terms  of  a  second 
order  Taylor  series  expansion  around  s  =  Q  where  the  constant  and  the  first  order  terms  are  zero, 
and  the  second  order  term  is  given  in  Eq.  19. □ 

Thus,  these  two  facts  (Eqs.  16  and  17)  together  show  that  both  the  average  and  average 
dc  are  locally  Euclidean  metrics.  However  we  recognize  that  for  small  signal  amplitudes,  the  error 
exponent  for  the  minimum  Pe /mini-max  detector  is  one  fourth  of  the  error  exponent  of  the  NP 
detector.  Thus,  for  small  signal  energies,  this  then  requires  that  the  minimum  probability  of  error 
detector  utilize  four  times  as  much  energy  as  the  Ney man- Pearson  detector  to  obtain  the  same 
performance  (as  measured  by  the  error  exponent.) 

This  small  signal  generalization  implies  that  the  local  performance  under  the  three  ciiteria  of 
interest  depends  only  on  the  signal  energy  and  not  on  the  specific  waveform.  As  such,  in  some  sense, 
this  result  demonstrates  that  non-Gaussian  environments  behave  as  Gaussian  environments  when 
the  signal  energy  is  small.  One  might  claim  that  this  is  merely  the  case  because  we  are  allowing 
the  length  of  the  data  vector  to  grow  without  bound  and  as  such  the  Central  Limit  Theorem  would 
apply.  However,  we  will  show  conclusively  that  this  is  not  the  case  for  all  energy  constraints. 


4.4.  Large  Signal  Behavior 


As  opposed  to  the  results  in  the  small  signal  case,  we  will  show  that  the  large  signal  performance 
depends  explicitly  upon  the  tail  of  the  noise  distribution  of  choice.  To  begin,  as  was  shown  in  Ref. 
1  that  for  large  signals,  the  Kullback-Leibler  distance  is  well  approximated  by  the  negative  of  the 
logarithm  of  the  density  function.  To  be  more  precise,  it  was  shown  that 


Urn 

S--+00 


dhljs) 

-logp„(s) 


=  1 


for  noise  densities  satisfying  some  very  general  conditions.  Similar  large  signal  results  can  be 
demonstrated  for  the  Chernoff  distance.  To  begin,  we  supply  an  upper  bound  which  holds  for  the 
same  general  class  of  distributions  considered  in  Ref.  1. 


Fact  4 

..  dcjs)  _  dcjs)  1 

dmis)  ~  -logpn(s)  ”  2' 


The  proof  is  based  on  Jensen’s  inequality. 

This  fact  suggests  that  for  large  signals,  the  error  exponent  for  minimum  Pg/mini-max  detectors 
can  be  no  bigger  than  one  half  of  the  error  exponent  for  NP  detectors.  As  described  in  the 
introduction,  this  should  be  the  case  since  the  NP  detector  need  only  minimize  Pm  while  the 
minimum  Pe  detector  must  simultaneously  minimize  Pp  and  Pm  and  therefore  can  commit  no 
more  than  one  half  of  the  computation  capability  of  the  likelihood  ratio  test  to  either  of  the  two 
error  probabilities. 
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If  we  limit  our  consideration  to  the  class  of  generalized  Gaussian  density  functions,  i.e.,  densities 
of  the  form 


Pn{x)  = 


1 


2r(l  +  iM(r) 
then  we  may  offer  much  stronger  results. 


exp 


1  /’ 


Fact  5  Let  the  noise  be  modeled  as  an  arbitrary  generalized  Gaussian  density  function.  Then 

i-*oo  -logp„(s)  [2  V  r  <l 


The  proof  is  shown  in  Appendix  C. 

Hence,  for  this  case  we  may  consider  large  signal  approximations  to  dc{s)  as: 


if  r  >  1  and  s  >  1 
if  r  <  1  and  s  >  1 

By  comparing  the  large  signal  results  presented  here  to  those  derived  in  [1],  we  observe  that 
the  error  exponent  (performance)  for  all  three  optimality  criteria  is  determined  by  the  quantity 
-  logp„(s),  which  in  the  generalized  Gaussian  environment  is  equivalent  to  an  £r  metric  of  value  |s|’'. 
Thus,  for  both  large  and  small  signal  energies,  the  dc  and  d/cL  are  identical  up  to  a  multiplicative 
constant.  However,  it  should  be  pointed  out  that  this  constant  diminishes  exponentially  fast  as 
the  decay  rate  of  the  density  increases.  This  implies  that  for  a  fixed  energy  level,  the  relative 
performance  of  the  minimum  Pe/mini-max  detector  as  compared  with  the  NP  detector  falls  off  at 
a  rate  of  1/2’’  in  the  performance  exponent  as  decay  rate  increases.  Nonetheless,  if  we  combine 
this  result  with  the  small  signal  results,  we  see  that  in  both  regimes  any  signal  that  optimizes  the 
Ney man- Pearson  performance  also  optimizes  the  performance  as  measured  by  minimum  and 
mini-max  criteria. 


{  log  Pn(») 

log|.(s) 


5.  SIGNAL  WAVEFORM  DESIGN  FOR  THE  NON-GAUSSIAN  CHANNEL 

For  the  family  of  generalized  Gaussian  noise  models  indexed  by  r,  we  consider  the  practical 
problem  of  allocating  the  available  energy  on  the  samples  of  the  signal  so  as  to  minimize  the 
three  performance  criteria  of  interest.  Without  any  available  analytic  solutions,  we  rely  upon  the 
asymptotic  relations  presented  in  the  generalizations  of  Stein’s  Lemma  and  Sanov’s  Theorem.  To 
accommodate  this,  we  pose  the  signal  design  problem  in  the  following  way:  Let  s^  be  a  length 
N  signal  vector.  Further,  let  sa/  be  the  N  x  M  length  signal  formed  by  repeating  s^  precisely 
M  times.  We  seek  to  determine  the  signal  waveform  sm  or  equivalently  s^  subject  to  an  energy 
constraint  such  that  the  three  performance  measures  of  interest  are  minimized  as  M  — >  00.  Note 
that  we  have  moved  from  the  original  problem  of  an  signal  of  finite  energy  to  a  power  signal,  so 
that  the  energy  constraint  E  on  s^  becomes  the  power  contraint  on  s^ .  For  simplicity  we  use  the 
same  notation  E  for  the  power  constraint. 
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Based  upon  this  formulation,  we  know  from  our  asymptotic  analysis  that  the  optimal  signal 
subject  to  the  NP  criterion  is  determined  by  maximizing  the  quantity 

s.t.  <  £. 

t=i  t=i 

Alternatively,  we  have  shown  that  the  optimal  signal  waveform  subject  to  the  minimum  and 
mini-max  criteria  is  determined  by  maximizing  the  quantity 

,  N  N 

S.t. 

t=l  »=1 

When  the  signal  energy  is  small,  we  know  from  our  analysis  in  Section  4  that  any  waveform 
satisfying  the  energy  constraint  will  be  optimal  for  all  three  optimality  criteria.  However,  if  we 
consider  the  large  signal  regime,  we  show  in  this  work  that  the  optimal  signal  must  be  either  fully 
impulsive,  i.e.,  sj  =  v^,  s,-  =  0  for  i  =  2,  or  constant  for  all  i,  that  is  Si  —  \/E/N  for  all  i. 

The  optimal  signal  s^  must  maximize  the  Lagrangian 

J=^E^(si)  +  p(E-^s^),  (20) 

^  .=1  t=i 

where  the  subscript  on  d(s,)  has  been  left  ambiguous  to  account  for  both  dc  and  Hki-  To  find  the 
maximizing  {sj}  for  J,  we  set  its  gradient  w.r.t.  to  zero. 

0  =  [VJ],  =  -  2pSi,  i  =  1,  ...N  (21) 

where  d(s,)  is  the  derivative  of  d(s,)  with  respect  to  Sj. 

Equation  21  can  be  shown  to  have  only  two  solutions,  namely  s,  =  0  and  s,  =  s*,i  —  1, ...,  iV, 
for  some  s*  ^  0,  as  demonstrated  in  Fig.  2  for  the  limited  set  of  pdfs  for  which  we  can  numerically 
evaluate  the  distances. 

The  maximizing  signal  set  s^  therefore  must  have  the  form  Si  =  s/Ej L  for  i  =  l,...,i  and 
zero  for  i  =  L-\- 1, ...,  N  for  some  L  satisfying  1  <  i  <  iV. 

We  pause  for  a  moment  to  consider  the  practical  implications  of  this  model.  First  for  the 
samples  i  -|- 1,  ,  the  received  signal  is  identical  under  either  Hq  or  H\,  and  more  importantly, 

this  fact  is  known  to  the  detector  that  will  disregard  samples  in  this  interval.  That  raises  the  second 
question,  namely  that  if  only  L  <  N  samples  are  ever  used  to  represent  the  binary  symbol,  why  not 
omit  N  from  the  problem?  The  answer  is  to  vary  N  would  amount  to  changing  the  symbol  rate 
of  the  problem,  and  in  turn  the  power  E  of  the  signal,  thus  making  any  performance  comparison 
meaningless. 

The  solution  to  the  minimization  problem  of  Eq.  20  is  most  succinctly  stated  through  the 
following  proposition  and  its  validity  may  be  readily  verified  for  the  Generalized  Gaussian  family 
of  pdfs,  but  it  is  rather  complex  for  the  general  case. 
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d  d_C(s)/ds, 


d  dKL(s)/ds, 


Fig.  2  —  Derivative  of  Chemoff  and  KL  distances  w.r.t.  displacement  r  vs  j 
intersects  any  linear  function  through  the  origin  at  most  one  more  point 


Proposition  8  Let  Pnix)  be  an  element  of  the  generalized  Gaussian  family  of  density  functions. 
Then  L  =  N  for  r  <2  and  L  =  I  for  r  >  2  under  each  of  the  three  optimality  criteria  of  interest. 

To  demonstrate  this,  consider  Figs.  3  and  4.  In  these  two  figures  we  have  plotted  the  average 
ChernofF  distance  and  the  average  Kullback-Leibler  distance,  respective!}',  as  a  function  of  L  for 
various  levels  of  power  E  G  [0.1, 100]  for  the  case  oi  N  =  20.  When  the  noise  is  Laplacian  (r  =  1) 
or  Cauchy,  the  maximum  occurs  when  L  =  N,  while  for  generalized  Gaussian  noise  with  r  =  4,  the 
optimal  choice  is  L  =  1.  We  see  that  for  small  signal  energies,  as  for  Gaussian  noise,  the  average 
divergence  and  Chernolf  distance  are  essentially  invariant  to  the  choice  of  L.  This  is  to  be  the  case 
since  in  this  energy  regime,  only  the  energy  and  not  the  choice  of  L  determines  performance.  In 
addition,  one  can  observe  that  aside  from  the  scale  (1/2’’  for  large  E,  1/4  for  small  E)  the  “shape” 
of  these  error  exponents  is  essentially  identical;  this  verifies  the  strong  similarities  established  in 
the  previous  sections. 

Since  any  waveform  with  a  given  energy  is  optimal  for  small  amounts  of  energy,  we  then  have 
arrived  at  the  following  optimal  signal  design  procedure  for  the  generalized  Gaussian  channel:  If 
the  tails  of  the  noise  density  fall  off  faster  than  Gaussian  tails,  the  optimal  length  N  signal  in  both 
the  large  and  small  energy  regimes  is  an  impulse  with  amplitude  y/E.  If,  however,  the  tails  fall 
off  slower  than  Gaussian  tails  (e.g.,  Laplacian  noise)  then  the  optimal  length  N  signal  is  constant 
with  amplitude  \/E/N. 

To  demonstrate  the  generality  of  these  results,  we  consider  the  Cauchy  density  as  an  alternate 
statistical  model  for  the  noise.  As  is  well  known,  this  model  has  polynomial  tails  and  as  such 
differs  significantly  from  the  generalized  Gaussian  model  considered  in  this  report.  In  addition,  in 
this  case  the  tail  of  the  Cauchy  density  diminishes  significantly  slower  than  those  of  the  Gaussian. 
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Gaussian:  (1/20)  L  C(sqrt(EA.))  Cauchy 


EOOl  EOOl 


Fig.  3  —  Average  Chemoff  Information  vs  the  number  of  nonzero 
signal  samples  L  for  a  variety  of  energy  levels  and  N  =  20 


Gaussian:  (1/20)  LdKL(sqtt(EA.))  Cauchy 


EOOl  EOOl 


Fig.  4  —  Kullback-Leibler  distance  vs  the  number  of  nonzero  signal 
samples  L  for  a  variety  of  signal  energies  and  N  =  20 
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Thus,  our  theory  would  suggest  that  constant  amplitude  signal  waveforms  should  be  optimal.  If 
one  considers  Figs.  3  and  4,  then  one  can  see  that  this  is  in  fact  the  case  for  the  error  exponent 
with  respect  to  both  dc  and  dfCL- 

It  should  be  stressed  again  that  these  length  N  signals  are  optimal  only  when  the  waveform 
is  repeated  an  infinite  number  of  times.  Of  course,  this  is  never  the  case  in  practice.  Therefore,  we 
are  obliged  to  consider  the  performance  of  these  signal  waveforms  in  practical  settings. 

In  Fig.  5,  we  have  plotted  the  error  exponent  derived  from  simulations  for  the  minimum  Pg 
detector  as  a  function  of  L  for  various  levels  of  energy  where  one  period  (Af  =  1)  of  the  signal 
is  transmitted  in  an  iV  =  20  sample  waveform.  The  simulation  was  performed  on  a  Connection 
Machine  CM-5  for  the  present  set  of  density  functions.  The  simulation  details  are  documented  in 
Section  6. 

From  an  asymptotic  standpoint,  this  case  of  M  =  1  is  a  worst  case  scenario.  Nevertheless,  as 
one  can  see.  Fig.  5  is  strikingly  similar  to  both  Figs.  3  and  4  even  though  the  functional  forms  of 
dci  djiL  and  Pe  are  drastically  different  from  one  another.  This  seems  to  suggest  that  the  similarity 
between  dc,  d^L  and  the  minimum  Pe  is  much  stronger  than  that  offered  by  the  duality  principle 
of  Bradt  and  Karlin  as  discussed  in  Section  2  of  this  report. 

6.  SIMULATION 


The  objective  is  to  estimate  the  detection  error  rate  for  our  binary  detection  problem  (Eq.  1) 
for  equal  priors,  and  when  only  one  period  of  the  optimal  signal  is  used.  For  power  E  and  number 
of  nonzero  samples  L,  a  noise  sequence  is  randomly  generated  according  to  density  distribution 


Gaussian:  Empirical  Error  Rate 


Cauchy 


Laplaclan  Gen  Gauss  4 


Fig.  5  —  Empirical  Probability  of  Error  vs  the  number  of  nonzero  samples  of  the 
signal  waveform,  L,  for  a  variety  of  signal  to  noise  ratios,  =  20  and  M  =  1 
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function  Pn  of  one  of  four  varieties,  namely  Gaussian,  Laplacian,  Cauchy,  and  Generalized  Gaussian 
with  decay  rate  4.  In  the  basic  Monte  Carlo  simulation,  the  following  is  repeated  until  the  desired 
accuracy  is  reached: 

•  Each  of  the  L  samples  of  the  sequence  are  examined.  This  sequence  is  equivalent  to  a 
L  —  sample  waveform  received  at  the  output  of  a  noisy  channel  under  the  null  hypothesis  Hq 
(zero  signal  energy). 

•  Each  received  sample  is  transformed  by  the  log-likelihood  function  log  Here  p„  is 

a  formula  that  describes  a  probability  density  function.  p„_s  is  another  formula  for  the 
alternate  hypothesis  pdf,  translated  from  pn  by  the  amount  s  = 

•  All  L  transformed  values  are  added  and  the  sum  is  compared  to  a  threshold  0.  If  positive, 
the  error  count  is  incremented  by  1.  (This  is  because  the  present  sample  is  transmitted  under 
Ho.) 


The  empirical  probability  of  error  is  the  error  count  divided  by  the  number  of  experiments. 

Importance  Sampling 

As  the  empirical  probability  (error  rate)  becomes  very  small,  Monte  Carlo  simulation  requires 
an  excessive  sample  size — in  the  order  of  the  inverse  of  the  error  rate.  Instead  of  using  the  sample 
mean  and  s  =  0  signal  amplitude  for  Hq,  we  use  the  Importance  Sampling  technique.  For  a 
rigorous  treatment  of  the  subject,  the  reader  is  referred  to  the  book  by  Bucklew  [2],  and  the  paper 
by  Orsak[12]. 

The  algorithm  is  as  follows: 


•  L  samples  of  randomly  generated  sequence  are  added  with  a  bias,  Xi  —  n,  -I-  Si,  Si  = 


Each  (biased)  received  sample  Xi  is  transformed  by  the  usual  log 


where  s  = 


•  For  each  sample,  a  counter-bias  weight  Wi  is  also  calculated.  =  log  ^^^(x,). 

•  All  L  transformed  values  are  added  and  the  sum  compared  to  a  threshold  0.  If  positive,  the 
error  count  is  incremented  by  an  amount  exp(W^).  The  exponent  W  is  the  sum  of  the  Wi  of 
each  of  the  sequence  sample. 


Even  though  the  sequence  still  represents  the  waveform  received  under  Hq,  the  bias  places  the 
distribution  precisely  at  the  threshold  of  the  log-likelihood-ratio  test,  such  that  an  “error”  occurs 
with  a  probability  in  the  vicinity  of 
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Random  Number  Generators 

The  simulation  runs  on  a  Connection  Machine  CM5  and  uses  the  uniform  random  number 
generator  native  to  this  parallel  architecture.  The  Gaussian  generator  uses  a  simplified  version  of 
the  polar  method  [13].  The  Laplacian  and  Cauchy  generators  use  the  transform  method[13]. 

The  Generalized  Gaussian  index  4  generator  uses  in  addition  to  the  uniform  number  generator 
a  Gamma{\,2)  generator.  The  latter  uses  Berman’s  algorithm[14]. 

Simulation  Codes  are  documented  in  Appendix  D. 

7.  CONCLUSION 

We  have  considered  the  problem  of  waveform  design  in  a  non-Gaussian  environment  for  com¬ 
munication  applications.  As  is  well  known,  optimal  Ney man- Pearson,  minimum  error  rate,  and 
optimal  mini-max  solutions  are  analytically  unavailable  when  the  background  noise  deviates  from 
Gaussian.  In  an  effort  to  obtain  “good”  solutions,  we  have  adapted  Large  Deviation  based  ap¬ 
proaches  to  determine  the  asymptotically  optimal  signal  waveform. 

The  principal  result  contained  in  this  report  was  to  show  that  for  a  given  statistical  model  of 
the  background  noise,  one  signal  waveform  is  optimal  with  respect  to  each  of  the  three  optimality 
criteria  described  above.  This  result  holds  for  both  large  and  small  signal  energies  (amplitudes). 
Moreover,  we  have  been  able  to  obtain  the  precise  waveform  for  a  wide  class  of  non-Gaussian 
statistics  of  much  current  interest.  In  addition  to  the  above,  we  have  demonstrated  the  following 
specific  results: 

•  extended  Chernoff’s  theorem  for  the  non  i.i.d.  problem  where  the  translation  amount  is 
known; 

•  demonstrated  that  the  Chernoff  Information  and  Kullback- Lei  bier  distances  determine  the 
error  exponent  in  the  problems  of  interest; 

•  calculated  large  and  small  signal  approximations  for  each  of  these  statistical  distance  mea¬ 
sures; 

•  calculated  formulae  for  dc  and  d^i,  for  a  variety  of  non-Gaussian  statistics; 

•  showed  that  for  the  generalized  Gaussian  class  of  pdf,  maximizing  dju  also  maximizes  dc, 
and  vice  versa; 

•  showed  the  maximizing  signal  of  these  two  distances  is  an  asymptotically  most  favorable 
signal,  in  the  sense  that  it  also  asymptotically  maximizes  all  distances  in  the  Ali  Silvey  class; 

•  compared  energy  required  to  maintain  same  error  exponent  under  NP  and  Bayes  criteria; 

•  designed  the  optimal  signal  waveform  under  NP,  mini-max  and  Bayes  optimal  criteria; 

•  confirmed  by  simulation  the  effectiveness  of  the  asymptotic  theory  as  applied  to  the  finite- 
sample  problem. 
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Appendix  A 

DETAILED  PROOF  OF  CHERNOFF  THEOREM 


In  this  appendix  we  present  a  detailed  proof  of  Chernoff  Theorem  extended  to  the  case  of 
independent  but  not  identically  distributed  samples. 

Consider  the  following  binary  detection  problem  in  which  an  N- vector  is  sent  through  additive 
iid  noise  channel  in  one  case  with  zero  signal,  and  in  the  other  with  some  known  signal  = 
{s„i  =  1,...,  TV}. 

Ho  :  Xi  =  n, 

jT  .  V  _  .  itd 

H\  .  X{  —  7l,‘  T  'S| ,  71, ‘  ~  p„ 


The  optimal  detector  computes  the  LLRT  based  on  the  aggregate  of  N  samples  and  compares 
it  to  some  threshold  7: 


(A-2) 


The  decision  regions  are  denoted  Zq  and  Z^: 

ZN  {X^|£(A^)>7} 


(A-3) 


The  false  alarm  and  miss  probabilities  are  defined  as  aj^  —  Pr{soj/  H\\Ho)  and  /Jat  =  Pr{say  Ho\Hi}, 
respectively.  Define  the  normalized  convex  combination  distribution  p\  as 


lpl-Hz^)p^izndz^ 


(A-4) 


For  the  additive  iid  noise  of  Eq.(  A-1),  under  either  hypotheses,  ihe  difference  between  the 
received  signal  and  the  transmitted  signal  {X^  —  s^)  =  {3:^  -  5,|i  =  1, . . A).  Each  term  Xi  —  5, 


is  Pn-  Thus 


Piix^)  =  nili  P\ix.)  =  nili  =  n.=i 

where 

J\{si)  f  Pi~^(^i)Pn(^i  -  Si)dZi 


(A-5) 


The  following  statistics  may  be  defined  in  terms  of  the  divergence  between  the  distributions  of 
the  hypotheseses  po  and  pi  and  the  normalized-convex  distribution  p\: 


Go,x(X^)  ^Eililog^ 

Gyx(X^)  ^j:N^log5^^ 


(A-6) 
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POtPi  may  be  expressed  in  terms  of  G  and  p\  as  follows: 

p(X^)  =  expiNGoA^^'mi^'') 

=  exp(7V(;o.A(Jf'^))n.P\(^.) 

=  exp{NGi,x{X^))pi(X^) 

=  exp(JVG,.A(A'^))n.p\(x.) 


(A-7) 


Observe  that  under  P)^(X^),  S{[og  is  identified  as  the  negative  of  the  Kullback- Lei  bier 

distance  d-Kii  so  that: 


^{Go,a(X«))  =  =  hZl,  -dKL(A,P.) 


N  ^i= 
1 


(A-8) 


By  the  Strong  Law  of  Large  Numbers,  each  of  the  statistics,  Eq.(  A-6)  approaches  its  average 
defined  in  terms  of  the  Kullback- Lei  bier  distances.  Therefore,  the  sample  average  of  the  statistics 
must  approach  the  sample  average  of  the  KL  distances. 


lim;v_oo  (7o,a  =  limA^-.oo;^Eili -‘^KiCPA’Pn)  a.s. 

lirnyv— »oo  f^l.A  —  limAT— Aco  ^E«=l  Pn-Si )  Cl-if- 


(A-9) 


Define  the  e  -neighborhoods  of  po  and  of  pi  in  terms  of  the  statistics  G  and  their 


averages: 

A"  I?  {X"  :  |<3„.a(X"')  +  E£i  Pn)l  <  <} 

B"  •W  :  IGtMX")  +  iEH,  dKL(pi,p.-..)l  <  £) 
As  N  -*  oo,  by  (A-9)  we  must  have  that  Vc,^  >  0,3iVo  such  that  for  ViV  >  Nq, 

Pr(A^|X^  ~  p^(^^))  >  1-6 
Pr(B^lX^  ~  p^(A^))  >  1-^ 


(A-10) 


(A-11) 


We  can  now  relate  the  false  alarm  and  miss  rates  to  the  c-neighborhoods  of  po  and  pi  as 
functions  of  e  and  the  Kullback- Leiber  distances  under  probability  measure  p^. 


>  sJ.nA^pMX^)e^<^oA^yx'^ 

>  fz^'nA^  p\iX^)e-^^-^L  'i^LiP\>Pn)dX^ 


(A-12) 


As  N  —*■  oo,  we  sdready  showed  that  Go.aC-A^)  approaches  -^Ei=i  ^KL{p\-,Pn)-,  i-e: 

1  N 

€  <  |Go,a(X'^)  a  -Y.dKL{p\.Pn)\  <  e 


(A-13) 


Continuing  developing  inequality  (A-12),  we  have 
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>  e-^‘-Er=i  ‘'^^i(PA'P")[p^^(Zi^)  +  Pa(^^)  -  1]> 


(A-14) 


where  the  last  inequality  is  from  P{Z  fl  A)  =  1  -  P{Z  U  A)  >  1  -  P{Z)  -  P{A)  =  1  -  (1  -  P[Z))  - 
(1  -  P{A))  =  P{Z)  +  P{A)  -  1,  and  the  assumption  that  P(Zi{X^))  >  |  with  A'"  distributed 


under  . 


The  first  Neyman  Pearson  lower  bound  is  then 


aN  >  ‘iKL(p\.Pu)^L  +  1  _  ^  _  1] 

^logOA^  >  -€  -  :^EiIl«iA-L{p\,Pn)+  ]^log(i  -  <5)  (A-15) 

limAT^oo  ^logOAT  >  limAr-.oo <^AL(PA-Pn) 


On  the  other  hand,  if  P{Z^{X^))  <  ^  under  this  distribution  then  we  have  P(Zq  (X^))  > 
and  a  similar  result  holds  for  the  miss  probability  as  follows. 


limA^^oo  log i3n  >  limA/-.oo  <^A'L(p\,Pn-5, ) 


(A-16) 


Combining  Eqs.  (A-15)  and  (A-16),  we  have  the  following  inequality  that  can  be  minimized 
with  respect  to  A  to  achieve  a  minimax  solution  for  Pe. 


1  ^  N  N 

— log(max(aAf,/3;v))  >  Ji^ min(^  dK-L(pt,p„),^  dK-L(p^Pn-s,))  (A-17) 


N-^oo  N 


Now,  the  minimum  probability  of  error  the  best  achievable  exponent  is  ‘^AA(PAo;Pn)» 
where  A,,  satisfies  : 

{N  N  A 

J2^KL{P\,Pn)  =  '^dKLip\,Pn-s,)  [ 

j=l  «=1  ) 


Hence, 


lim  minPe  >  lim  e  S>=i  “^^ilPxo’P") 


iiiii  (fvvrc'.i  g  xiiix 

iV— POO  N~-*oo 


(A-18) 


Before  proceeding  any  further  we  must  make  sure  that  the  assumptions  made  in  the  derivation  of 
equations  (A-15)  and  (A-16)  still  hold  simultaneously. 


Lemma  9 


argx  ( dKL{p\iX’^),p{X^))  =  dj^i^X^lpiX^  -  s^))  such  that  lim  £\[Iz^iX^)]  =  U 

TV— POO  1  Z  J 

=  argx  {d^MiX^piX^))  =  dKL{p‘x{X^),p{X^  -  s^))} 
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Proof:  For  minimum  Pe  define: 


The  likelihood  ratio  is, 


Therefore, 


Therefore  for  large  N, 


lim  CIX^)=  Um  ? 


f:,[izs{X^)]  =  P[X^  ez^\x^ 


=  Pix«|i|;iog{^}  >  o,.v«  ~  pii. 


Now  by  the  strong  law  of  large  numbers. 


UniN-oo^E.i,log{£ig{}  =  llm«_„i^E'l,/log{£jf|{}p'»,(a:,)rfx,  a.,.  (A-I9) 


Also  for  A  =  Afl 


=  -Y,dKL(p\„>Po)  +  ^dKL(p\,^P\] 


Therefore, 


Hence, 


limyv-.co^Er=ilog|^|  =  0  o.s. 


e.[Mx^)]  =  e,[izs{x^)]  =  -. 


(A-20) 


The  lemma  shows  that  we  still  satisfy  the  assumptions  in  arriving  at  Eqs.  (A-15)  and  (A-16). 
Hence  from  Eq.  (A- 18)  , 

1  I  ^ 

lim  —log min >  lim  --j;j'^dfcL{Px^,Pn)- 
N-*oo  N  N-^oo  N 


(A-21) 
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Chemoff  Information 


Lemma  10 


Proof 


1  ^  1 
H =  min  —  log  Ja(s^). 

1=1 

log  =  log  J p^-\X^}p\X^  -  s^)dX^. 


Since  the  components  of  X^  are  independently  distributed. 


logJA(s^)  =  Y^log  J ^(xi)p^(x,  -  s,)dx, 

=  l^log y 


Majcimizing  Jx  wrt  A: 

d  ^  I  f 

^log  Ja(s^)  =  J  P^~^{xi}p^{xi  -  s,)[logp(i,  -  5.)  -  logp(i,))di,. 

N 

=  Yl^-(^KLip\,Pn-,,)  +  dKliPx^Pn)]- 

i=l 

which  mu^t  be  equal  to  0  for  minlog  Ja('^^)-  Hence  the  minimum  Ao  is  given  by, 

N  N 

X!®^A'L(PA„iPn)  =  J2dsL{p\^..Pn-s,)- 
i=l  1=1 

Now  to  get  miuA  ^log  Ja(-^^),  we  must  proceed  as  follows.  Consider; 

logp^A,,!^"^)  =  (1  -  A)logp(X'^^  +  Alogp(X^  -  s^)  -  log 


Therefore, 


.  J  r  N.  XM  P(^^)  M  pIX^-s^) 


Taking  the  expectation  value  w.r.t.p^^(X^): 


log  =  -(1  -  A)dKL(pt(X^),p(A''))  -  AdKL(pL(X^),p(X^  -  s^)) 

N 

=  -'^dKL{p\,{x,),p{Xi)) 


(A-22) 
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by  making  use  of  the  fact  that  Aq  results  in  the  minimum  value  for  Jx{s^}  .Hence  we  have, 

N  N 

-  log  Jx,{s^)  =  X!rfK-L(p\,Pn)  =  ^dKL{p\,Pn-s,)- 

«=1  t=l 

Hence  the  result. □ 

FVom  Eq.  (A-21)  we  have, 

lim  -^logminPj^  >  lim  ■^logJA„(s^) 

N—*oo  IS  N— *•00  iV 

which  is  the  desired  lower  bound  for  minPe. 

Upper  bound,  Chemoff 

We  will  now  show  an  upper  bound  for  minPe  which  approaches  the  value  of  the  lower  bound, 
thus  establishing  the  asymptotic  equivalence  of  the  Chernoff  Information  dc  to  the  best  annihilating 
rate  for  Pg. 

min  P^  =  /  min  (iroPniX^),  iriPn{X^  -  s^))dX’^ 

<  I  i^OPniX^)y-\TiPn{X^  -  s‘^))^dX^ 

=  nlli  SpI  (^i)Pni^i  -  Si)dxi  (A-23) 

=  •n-o~^’ri^exp(i:iog/p^-''(x,)p^(xi  -  Si)dx<) 

^logminPf  <  ^log(T^"V)+ 

Taking  the  limit: 


lim  ^logminPi^  < 
N—^oo  IS 


Jta  mini  log  *(.") 


s,  ^dx{ 


(A-24) 


Take  the  upper  and  lower  bounds  together,  we  have  the  limit. 


Appendix  B 

DIRECT  CALCULATION  OF  CHERNOFF  INFORMATION 


In  this  appendix  we  derive  the  Chernoff  Information  for  the  Gaussian,  Cauchy,  Laplacian  and 
Generalized  Gaussian  of  index  4  densities.  The  results  are  summarized  in  Table  1. 


Gaussian 


Probability  density  function: 


Cois^cr}  =  -log/^^exp-^(x2  +  (x-s)2)d2: 

=  -log/:75^exp-24(x  -  f)2  + (f)2)da 


S* 

8^ 


Cauchy 


Probability  density  function: 


Cc(a,(r) 


1 


dx 


dy 

dy 


(B-i: 


From  Ref.  Bl,  Formula  266.00,  and  definitions  and  modulus  properties  of  T  in  pages  xv,  12: 


D{a,  a) 
where 

T{-x,k) 


=  ,  »  dy  =  ^^(7r,A:) 

y/y*+Py^+a*  ^  2a,  y  '  > 


2JP'(|,fc) 


(B-2) 


/■(|,  k)  is  the  complete  elliptic  integral  of  the  first  kind  that  we  denote  in  Table  1  of  the  report 
as  Ek- 
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The  application  of  Formula  266.00  requires  that 


£^"  +  77:  + 


1 


2  '  V  2  '  ’  16  a- 

This  caji  be  shown  to  be  true  for  all  positive  values  of  a  and  ct. 
Apply  Eqs.  (B-2)  to  (B-1),  we  have  the  substitutions 


13  =  2o2-4 

a‘*  =  ^ 

*  /t2  X  A  ,  1 

+  16  + 

s£_ 

k  = 

*  —  2  ^ 

\ 

y  2  ^  ^  16 

so  that 

C(s,a) 

where  k  is  defined  in  Eq.  (B-3). 
Laplacian 

Probability  distribution  function: 

CLis,(T) 


iog^D{s,(T) 

logv-1 - 1 - 


d4yg 


=  -  log[e2<'/v^  -t-  -^e  2°7^] 


+  XT' 


Generalized  Gaussian  index  4 

Probability  distribution  function:  '''^here  A(r)  =  ]' 

Cgg4(s,(t)  =  -logf^^  pi(x}p^_^xdx 

1 


y=x-^ 


2r(fM7 


1 


-log 

-  2fTf)A4  *'  >0  2A\ 

log  r(  1^4)+  -  log /o°°  exp  jriy'^  +  ^s'^y^)dy 


-2  exp -^(2y‘*  +  q.  §L)dy 


(B-3) 


(B-4) 


(B-5) 
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The  argument  of  the  logarithm  of  the  third  term  in  Eq.(  B-5)  is  evaluated  using  an  identity  in 
Ref.  B2.  The  identity  is 


exp  -  2'y^x^dx  =  2 


2/?2' 


(B-6) 


where  A'l  is  the  modified  Bessel  function  with  fractional  order  We  apply  identity  Eq.  (B-6) 
with  and  7  = 


The  integration  is  then 


1  3 

exp  -I-  -s‘^y^)dy  = 


Ail) 

32  A\ 


(B-7) 


Substituting 


C(a,<T) 


\/5^ 

32  A* 


9  s'* 

log  A'l  ( - -) 

^  *^32  A‘*’ 


(B-8) 


where  A  =  A4  = 
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Appendix  C 

LARGE  SIGNAl  APPROXIMATION  FOR  CHERNOFF  INFORMATION  FOR 
AN  EXPONENTIAL  FAMILY  OF  PROBABILITY  DENSITIES 

In  this  appendix  we  detail  proofs  for  Fact  5  in  the  main  text  that  provide  upper  bounds  to  the 
Chernoff  Information  for  large  signal.  It  is  repeated  here: 

Consider  densities  of  the  form  p„(a:)  =  exp  {-  where  A{r)  = 

Then  if  Z  denotes  the  following  limit: 

2  =  lim  =  um 

a-oo  _logp„(5)  5-00  -logp„(s) 

it  must  be  that  : 

Proof;  Probability  densities  of  this  form  are  known  as  Generalized  Gaussian  density  with 
decay  rate  r  and  variance  cr^.  We  begin  with  the  r  >  1  case. 

1  For  r  >  1  ; 


Dividing  the  limits  of  the  integral  involved  in  C(s)  as  follows 


/oo  rO  ,  ri 

p\!\x)p\l\x  -  s)dx  =  I  p]/\x)pl/\x  -  s)dx  +  I  pI/\x)pI/\x  -  s)dt 
'OO  J—oo  Jo 

rco 

+  Pn'^{x)pl/\x  -  S)dx 


(C-1) 


.  Now  denoting  6  =  A{ry  and  K,  =  ,  ,  the  second  term  on  the  RHS  of  the  above 

equation  simplifies  to  , 


p]J'^{x)p]l\x  -  s)dx  =  /Cjf  exp 

^  r _ + 


where  the  inequality  was  introduced  by  using  the  relation:(Ref.  Cl) 


l®r  +  |a:-ar  > 


{N+ix-sir 


(C-2) 
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which  holds  for  all  r  >  1  and  for  all  x  and  s.  Thus,  by  recognizing  that  |a:|  -|-  |x  -  s|  =  s  over 
the  region  of  integration,  we  have  that 


/  pI/^{3:)pI/\x  ~  s)dx 
Jo 


<  K  r  exp 

=  /Csexp-;^. 
^  2^b 


(C-3) 


Proceeding,  we  turn  our  attention  to  the  first  term  in  Eq.  C-1  to  establish  an  upper  bound 
for  the  integral  as  follows  : 


J_^Pn\x)p]/^{x  -  s)dx  =  1C  exp  exp  -  dx. 

Now  the  term  |exp  maximum  value  at  x  =  0  for  x  G  [-oo,0],  hence  we  may 

upper  bound  the  integral  by, 

/•oo  ~  ^ 

= 


where  the  latter  integral  was  evaluated  by  using  the  fact  that  /Cexp  — ^  is  a  valid  density 
function.  Similarly  we  may  upper  bound  the  third  term  in  Eq.  C-1  as  follows: 


^  Pn\x)plf'^{x  -  s)dx  -  K.  exp  exp  ■  ■  dx 

Here  the  term  |exp  has  its  maximum  value  at  x  =  s  for  x  6  [s,oo].  Hence  we  can 

upper  bound  the  integral  by, 

where  the  latter  integral  was  evaluated  as  before.  Therefore  by  combining  the  upper  bounds 
on  the  terms  in  the  RHS  of  Eq.(  C-1)  we  arrive  at  the  following  upper  bound  , 


j_  pyV)py^(2^-«)da;<2*/’’exp|-|^|  +  /Csexp-J^.  (C-6) 


Thus  it  must  be  that 
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-  log{/-!^Pn  -  s)dx} 

-logPn(s) 

Therefore  from  Eq.  C  , 


-  log{2^/*'  exp  +  Ks  exp 
-logAC  +  |sl’'/6 


Z  =  lira  ~ 

j-oo  -logp„(;8) 


>  lim 

a 


im  < 

-♦OO  I 


-  log  {exp  [-y^]}  log{/Cs  +  2i/’-exp  ^ 


\s\r/b  \s\r/b 

Now  as  s  -♦  oo  ;  exp  [-^  +  — »  0  since  r  >  1,  so  the  above  limit  reduces  to, 


1 


(C-7) 


thus  establishing  a  lower  bound  on  Z.  Now  we  seek  to  bound  Z  from  above  by  the  same 
quantity  and  we  proceed  as  follows. 

We  know  that  : 


i«r  +  \x-  -sr  <•*’■- • 


for  X  e  [0,a].  Hence, 


/  Pn\^)Pn\x  -  s)dx  =  2  /  ^  p}/^{x)pI/\x  -  s)d. 
Jo  Jo 


/•s/2 

>  2  /  Ke  25  *  dx 

Jo 


=  2Ke-^ 


,r  y»/2 


I 


26  dx 


2/Ce  2E  f  I 


Therefore  we  have, 

/OO  f8 

Pn\x)pl/\x  -  s)dx  >  I  p]l\x)p]l'^{x  -  s)dx. 

-OO  Jo 

2Ke-i_  f  I 


Z  <  lim 


|->og|j^ 


2K 


-loge" 


26 


26  —  log  <  e  26 


Therefore, 
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=  lim 


£l  _  crfl _ 1 

26 _ iii _ 


Hence  combining  the  upper  and  lower  bounds  of  Eqs.  C-7  and  C-8  we  get  the  desired  result. □ 
2  For  r  <  1  : 

Let  us  start  out  by  establishing  a  general  upper  bound  valid  for  all  probability  densities.  We 
shall  first  upper  bound  the  ChernofF  information  C(s)  using  Jensen’s  inequality  : 


dcis)  =  -  !og{  J  -  s)dx} 

-  -“•('■('Sefll 


Thus  it  is  sufficient  to  show  that, 

I-  dKiis)  . 

urns  00 — z - -  =  1. 

-logPn(«) 

Proceeding:  we  may  write  the  above  ratio  as 

dh-ljs)  _  f  {log{pn(a:)  -  logPn(a:  -  .s)}}  Pn{x)dx 
-logPn(s)  -logp„(s) 

The  final  result  is  readily  established  by  making  the  following  assumptions 

(a)  Pn(a:)  is  symmetric  in  the  tails, 

(b)  lim,_*oo  =  0  for  all  x, 

(c)  lim.^oo  =  1  for  all  x. 


(C-9) 


(C-10) 


to  arrive  at: 


Z  <  lim  s  — >  oo 


[  1  dKijs)  ]  ^  1 
l2{-logp„(s)}  /  2 


Thus  establishing  the  desired  result.  As  before,  we  shall  lower  bound  Z  by  the  same  quantity 
namely  Here  we  make  use  of  the  fact  that  : 


xp  +  lx  -  sr  >  |s| 


for  X  €  [0,s].  Therefore, 


f  pI/^{x)pI/^{x  —  s)dx  <  fC  f  dx 

Jo  Jo 
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also  from  Eqs.  (C-4)  and  (C-5), 

-  s)d.  < 

and, 

^  Pn^{x)p\!\x  -  s)dx  < 

Therefore, 


/OO 

Pn^(x)pi^^(x  -  S)dx  < 

-OO 


Hence, 


2  > 


Jim 

1 

2‘ 


By  combining  the  upper  and  lower  bounds  together  we  get  the  desired  result. 
If  we  consider  only  the  Generalized  Gaussian  density  function,  we  must  have: 


For  r  >  1: 

and  for  r  <  1: 

for  large  ’s’. 


(C-11) 

(C-12) 
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Appendix  D 

CM  FORTRAN  SIMULATION  CODES 


program  simulation 

integer  NtMax,  NMAX.  MMAX.  EMAX.  SMAX 

parameter  (NMAX=  256,  N2MAX=  256,  MMAX=20,  EMAX=  7) 

C  NtMAX:  meuc  number  of  blocks  of  time  series  data 

C  NMAX:  time  series  length;  actually  a  slice  small  enough  to  fit  machine 

C  N2MAX:  second  time  series  length  to  do  multiple  slices  in  parallel 

C  MMAX:  maximum  number  of  channels 

C  EMAX:  maximum  number  of  Energy  level? 

Real  pO,pl  !  apriori  prob  of  HO  to  HI 

Real,  array(EMAX),  DATA  : :EE  =  [0.1,  1.0,  10.0,  30.0,  50.0,  75.0,  100.0] 
Real  N(NMAX,  N2MAX,  MMAX)!  time  series  of  noise,  variance  1  noise 

!  iid  among  samples  and  among  channels 
Real  W(NMAX,  N2MAX)  !  time  series  of  weighted  received  signal 

Real  WW(NMAX,  N2MAX)  !  time  series  of  weighted  received  signal 

Real  RCNMAX,  N2MAX)  I  time  series  of  received  data 

Real  L(NMAX,  N2MAX)  !  time  series  of  Bayesian  statistic 

Real  P(MMAX,  EMAX)  !  prob.  of  error 

Real  Ptemp(MMAX,  EMAX)  !  prob.  of  error 
Real  a,  temp,  s,  noise 
character* 10  PDF,  myPDF 

CMF$  LAYOUT  N(:serial,  :news,  :news),  L(:serial,  :news) 

CMF$  LAYOUT  R(: serial,  :news) 

CMF$  LAYOUT  W(: serial,  :new8) 

CMF$  LAYOUT  WH(: serial,  :news) 

CMF$  LAYOUT  P(:news,  :news) 

CMF$  LAYOUT  Ptemp(:news,  :news) 

Integer  ns,  ne,  i,  im,  m,  Ei,  ii,  j,  jj 

Integer  thePDF 

include  ’ Random . h ’ 

include  ’LogLRT.h’ 

include  ’cuigle. comments’ 

pO  =0.5 
pi  =0.5 

10  print* , ’Which  noise  PDF  ? 

read* ,PDF 
print* , PDF 

if  (whichPDF(PDF)  .EQ.  0)  then 
print*, ’hey’ 
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C 

C 


C 


C 


98 

99 


go  to  10 
end  if 

print* ,whichPDF(PDF) 

print* , ’Which  detector  PDF  ?  ’ 

read* ,myPDF 

print *,myPDF 

if  (whichPDF(niyPDF)  .EQ.  0)  then 
print* , ’hey’ 
go  to  11 
end  if 

print*, ’How  many  slices  (’,  NMAX*N2MAX,’  samples  each)?’ 

read*,NtMAX 

P(1;MMAX.1:EMAX)  =  0.0 

do  Nt  =1,  NtMAX  !for  all  time  slices  Nt  =  1 :  NtMax 

do  im  =  1,  MMAX 

M(1:NMAX.  1:N2MAX,  im)  =  MyRand(NMAX,N2MAX ,  PDF) 
end  do 

Ptemp(l:MMAX,  1;EMAX)  =0.0 
do  Ei  =  1.  EMAX 
EO  =  EE(Ei) 

do  m=l,  MMAX  !  m  is  the  grouping  index 

L(:,:)  =0.0  !  init  the  statistic  array 

a  =  Sqrt(E0/m) 

H(:,:)  =  0.0 

do  i=l,  m  !  channel  i 

R  =  a/2.0  +  N(;,;,i)  !this  line  for  Importemce  Sampling 

R  =  N(:,:,i)  !this  line  for  Monte  Carlo 

W  =  W  +  LogLRTCNMAX,  N2MAX,  R,  a/2.0,  PDF,  1)  !for  IS 
L  =  L  +  LogLRTCNMAX,  N2MAX,  R,  a,  myPDF,  1)  !for  both 

end  do 

WW(:,:)  =  0.0 

forall  (ii=l:NMAX,  jj=l:N2MAX,  L(ii, j j) .LT.Log(pl/p0)) 

1  WW(ii,jj)  =1.0  !for  Monte  Carlo 

1  WW(ii,jj)  =  ExpCWCii, j j))  !for  IS 

temp  =  sum(WW) 

PtempCm,  Ei)  =  temp 

Accumulate  the  empirical  prob  of  error  across  time  slice 
P(m,Ei)  =  P(m,Ei)  +  temp 
end  do  !  channel  grouping 
end  do  !  energy 
end  do  !time  slices 
P=  P/(NMAX*N2MAX*NtMAX) 
print* ,PDF , ’ - ’ ,myPDF 
write(*,98)EE(l:EMAX) 

formatClx,’  ml,  E->’ ,7(F16.4,  ’  ’)) 

do  m=  1,MMAX 

write(*,99)m,  P(m,  1:EMAX) 

format(lx,I4,TR1.7(G20.8,  ’  ’)) 
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end  do 
end 


include  ’Random. fern’ 
include  ’LogLRT.fcm’ 

C . 

C  File  LogLRT.fcm 

R,  a,  PDF,  s) 

!  vector  length,  number  of  cheumels 

!  received  vector,  translation,  std  dev 
!  which  pdf 

:news),  R(: serial,  :neus) 


function  LogLRT(NN ,N2 , 
Integer  NN.  N? 

Real  LogLRT(NN,N2) 

Real  R(NN,  N2) ,  a,  s 
character* 10  PDF 
CMF$  LAYOUT  LogLRT( : serial , 


integer  i 
Real  a2,  s2 

Real  AA2,  AA4,  GQuaurter,  GFiveQ,  GThreeQ 
interface 

integer  function  whichPDF(PDF) 
character* 10  PDF 
end  interface 


C  calculates  the  statistic  for  use  in  the  Optimal  test 

a2  =  a*a 
s2  =  s*s 

select  case  (whichPDF(PDF) ) 
case  (:0) 

LogLRT(l:NN,  1;N2)  =  0.0 
case  (1)  ! Gaussian 

LogLRT(l;NN,  1;N2)  =-(a/s2)*R(l :NN,  1:N2)  +  a2/(2.0*s2) 
case  (2)  ILaplacian 

LogLRT(l:NN,  1:N2)  = 

1  -(l/s)*Sqrt(2.0)*(Abs(R(l:NN,l:N2))  -  Abs(R(l :NN , 1 :N2)  -a)) 

case  (3)  ! Cauchy 

LogLRT(l:NN,l:N2)  = 

1  LogCd.O  +  ((R(1:NN,1:N2)  -a)/3)**2>/(l+  (R(l  :NW.l:N2)/s)**2)) 

case  (4:)  !GenGasuss4 

GQuarter  =  3.6256099082 
GThreeQ  =  1.2254167024 
GFiveQ  =  GQueu:ter/4.0 
AA2  =  s2*GQuarter/GThreeQ 
AA4  =  AA2*AA2 

LogLRT(l:NN,l:N2)  =  (1 .0/AA4)*((R(l :NN, 1 :N2)-a)**4  -R(l :NN, 1 :N2)**4) 
end  select 
return 
end  function 


38 


Chu,  Warke,  and  Orsak 


C  File  Random. f cm 

Function  RandUniform(N,N2) 

Integer  N,N2 

Real  RandUniform(N,N2) 

CMF$  LAYOUT  RandUniformC .serial, ;news) 

100  call  cmf_random(RandUniform(l:N,l;N2)) 

if  ((minval(RandUniform)  <=  0.0)  .OR.  (maxval(RandUniform)  >= 
goto  100 
end  if 
end 

Function  ReuidGaussian(N,N2) 

Integer  N,N2 

Real  RandGaussian(N,N2) 

Real  Temp (N, M2) 

Logical  ISHALFCN,  N2) 

CMF$  LAYOUT  RandGaussianC : serial,  .-news) 

CMF$  LAYOUT  Temp (: serial ,  :news) 

CMF$  LAYOUT  ISHALF( : serial ,  :neHs) 

integer  i,  j 
real  twopi 

twopi  =  4.0*acos(0.0) 

100  call  cmf.random(Temp(l:N,l;N2)) 

RandGaussian(l;N,l:N2)  =  SQRT(-2.0*Log(Temp(l:N,l:N2))) 

200  call  cmf_random(Temp(l;N,l:N2)) 

RandGaussian  *  RandGaussian*cos(TwoPi*Temp) 
end 

Function  RandLaplacian(N,N2) 

Integer  N,N2 

Real  RandLaplacianCN,  N2) 

Real  Temp(N,  N2) 

CMF$  LAYOUT  RandLaplacian(; serial,  -.news),  Temp(; serial,  -.news) 

Real  s2 
Integer  i,  j 

100  call  cmf _random(Temp(l :N, 1 :N2)) 

if  ((minval(Temp)  <=  0.0)  .OR.  (maxval(Temp)  >=  1.0))  then 
goto  100 
end  if 

s2  =  1.0  /Sqrt(2.0) 

RandLaplaciand :N, 1 :N2)  =  s2*Log(Temp(l:N,l:N2)) 

call  cmf_random(Temp(l:N,l:M2)) 

forall  (i=l:N,  j=l:N2,  Temp(i,j)  >  0.5) 

1  RandLaplacian(i, j)  =  -  RandLaplaciand, j) 


1 .0))then 
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end 

Function  RandCauchy(N,N2) 

Integer  N,N2 

Real  RandCauchy(N,N2) 

Real  Temp(N,N2) 

CMF$  LAYOUT  RandCauchyC:  serial,  -.zievs)  ,  Temp(  :  serial ,  rnews) 
Integer  i,  j 
Pi  =  2.0*acos(0.0) 

100  call  cmf_random(Temp(l:N,l:N2)) 

if  ( (minvaKTemp)  <=  0.0)  .OR.  (maxval(Temp)  >=  1.0))  then 
C  print*, ’bad’ 

goto  100 
end  if 

RandCauchyCl :N, 1 :N2)  =  tan(Pi*(Temp(l :N ,  1:N2)  -  0.5)) 
end 

Include  ’ Gamma. f cm’ 

Function  RandGenGau3s4(N,N2) 

Integer  N,N2 

Real  RandGenGauss4(N  N2) 

Real  Temp(N,N2),  X(N.N2) 

CMF$  LAYOUT  RandGenGaus84( : serial,  mews),  Temp( merial ,  mews) 
CMF$  LAYOUT  X(: serial,  mews) 

Real  A,  A4,  GQueurter,  GFiveQ,  GThreeQ,  alpha 
Integer  i,  j,  ii,  jj 
real  twopi 
interface 

function  RandGammal(N,N2,aa) 
integer  N,  N2 
real  RandGafflmal(N,N2) ,  aa 
CMF$  LAYOUT  RandGammal (: serial ,  mews) 

end  interface 

twopi  =  4.0*acos(0.0) 
alpha  =  1 . 0/4 . 0 
GQueurter  =  '‘.6256099082 
GThreeQ  =  1.2254167024 
GFiveQ  =  GQuarter/4.0 
A  =  Sqrt(GQueu:ter/GThreeQ) 

100  call  cmf _random(Temp(l :N, 1 :N2)) 

if  ( (minvaKTemp)  <=  0.0)  .OR.  (maxval(Temp)  >-  1.0))  then 
print*, ’bad’ 
goto  100 
end  if 

X(1:N,  1:N2)  =  RandGammal (N,  N2,  alpha) 
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RandGenGauss4  =  A*SQRT(SQRT(X)) 
call  cmf_random(Temp(l:M,l:M2)) 
forall  (ii=l;N,  jj=l:N2,  TempCii, j j)>0 .5) 

1  RandGenGauss4(ii, j j)  =  -  RandGenGauss4(ii, j j) 

end  function 

Fimction  MyRand(N,N2,  PDF) 

Integer  N,  N2 
Real  MyRandCN,  N2) 

Character*10  PDF 
Real  Temp(N,  N2) 

CMF$  LAYOUT  MyRandC : serial,  mews),  Temp(:serial,  mews) 

interface 

integer  function  whichPDF(PDF) 
character* 10  PDF 
end  interface 

interface 

function  RandUniform  (N,N2) 

integer  N,N2 

real  RandUniform(N,N2) 

CMF$  LAYOUT  RandUniformC -.serial,  mews) 

end  interface 

interface 

function  RandGaussian  (N,N2) 
integer  N,N2 
real  RandGaussian(N,N2) 
real  Tenip(N,N2) 

CMF$  LAYOUT  RandGaussianC: serial,  mews),  Temp( iserial,  mews) 

end  interface 
interface 

function  RandLaplacian  (N,N2) 
integer  N,N2 

real  RandLaplacian(N,N2) 
real  Temp(N,N2) 

CMF$  LAYOUT  RandLaplacianC; serial,  mews),  TempC :serial,  mews) 

end  interface 
interface 

function  RandCauchy  (N,  N2) 
integer  N,N2 
real  RandCauchy (N,  N2) 
real  Temp(N,  M2) 

CMF$  LAYOUT  RandCauchy  (-.serial,  mews),  TempC  :serial,  -.news) 

end  interface 
interface 

function  RandGenGauss4  (N,N2) 
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integer  N,N2 

real  Ran(lGenGauss4(N,N2) 
real  Temp(N,N2) 

CMF$  LAYOUT  Ran<lGenGauss4(: serial,  :nevs),  Temp( rserial,  :nevs) 

end  interface 

select  case  (whichPDF(PDF)) 
case  (:0) 

MyRand  =0.0 
case  (1) 

MyRand  =  RandGaussian(N,N2) 
case  (2) 

MyRand  =  RandLaplacian(N,N2) 
case  (3) 

MyRand  =  RandCauchy(N,N2) 
case  (4) 

MyRand  =  RandGenGauss4(N,N2) 
case  (5) 

HyRamd  =  RandUniform(N,N2) 
case  (6:) 

MyRand  =0.0 
end  select 
return 
end  function 

Function  whichPDF(PDF) 
integer  whichPDF 
character* 10  PDF 

if ( index (PDF , ’ Normal ’ ) . NE . 0 )  then 
whichPDF  =  1 

else  if (index (PDF, ’normal’) .NE.O)  then 
WhichPDF  =  1 

else  if (index(PDF, ’gaussian’) .NE.O)  then 
WhichPDF  =  1 

else  if (index(PDF, ’Gaussian’) .NE.O)  then 
WhichPDF  =  1 

else  if (index(PDF, ’Laplacian’) .NE.O)  then 
WhichPDF  =  2 

else  if (index(PDF, ’Laplace’ ) .NE.O)  then 
WhichPDF  =  2 

else  if ( index (PDF , ’ laplacian ’ ) . NE . 0)  then 
WhichPDF  =  2 

else  if (index (PDF, ’laplace’ ) .NE.O)  then 
WhichPDF  =  2 

else  if (index(PDF, ’Cauchy’ ) .NE.O)  then 
WhichPDF  =  3 

else  if (index(PDF, ’cauchy’ ) .NE.O)  then 
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whichPDF  *  3 

else  if (index(PDF, ’GenGauss4’ ) .NE.O)  then 
WhichPDF  =  4 

else  if (index (PDF, ’gengauss4’ ) .NE.O)  then 
whichPDF  ®  4 

else  if (index (PDF, ’Gengauss4*) .NE.O)  then 
WhichPDF  *  4 

else  if (index (PDF, ’uniform’) .NE.O)  then 
whichPDF  =  5 

else  if (index (PDF, ’Uniform’) .NE.O)  then 
whichPDF  =  5 
else 

whichPDF  =  0 
end  if 
return 
end  function 

Function  erf(NHAX,  N2MAX.  R,  x) 
real  erf 

integer  NMAX,  N2MAX 
real  R(NHAX.  N2MAX) 
real  x 

CMF$  LAYOUT  R( '.serial,  '.news) 

real  One (NMAX,  N2MAX) 

CMF$  LAYOUT  One( '.serial,  :news) 

integer  ii.jj 

One( : , :)  =  0.0 

forall(ii=l:NMAX,  jj=l:N2MAX,  R(ii, jj) .LT.real(x)) 

1  0ne(ii,jj)=1.0 

erf  =  sum(0ne)/real(N2MAX*NMAX) 

return 

end  function 

C . 

C  File  Gamma. fern 

Function  RandGammal(N,  N2,  aa) 

Integer  N,N2 

Real  RandGammal(N,N2) 

Real  aa 

Real  Temp(N,N2) 

Real  Temp2(N,N2) 

Real  X(N,N2) 

Real  Y(N,N2) 

Real  Z(M,N2) 

CMF$  LAYOUT  RandGammaK : serial,  ;news),  Temp( : serial ,  rnews) 

CMF$  LAYOUT  Temp2( .'serial ,  :news) 
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CMF$  LAYOUT  X(: serial,  rnews) 

CMF$  LAYOUT  Y(: serial,  tnews) 

Integer  i ,  j ,  retry 
Real  b 

retry  =  0 

C  print*, ’aa’,  aa 

b  =  1.0/(1.0-aa) 
call  cmf  .random (Tempd  :N,  1;N2)) 

X(:,  :)  =  Temp(:,  ;)**(1.0/aa) 
call  cmf _random(Temp(l :N,  1:N2)) 

Y(;.  :)  =  X(:,  :)  +  Tamp(:.:)**b 

100  if  (maxval(Y) .GT.1.0)  then 
retry  =  retry  +  1 

if  (modCretry , 100)  .EQ.O  )  then 

C  print*, ’retry  =’,  retry 

endif 

call  cmf .random (Temp2(l ;N,  1;N2)) 
call  cmf_random(Temp(l :N,  1:N2)) 
where  (Y( : . : ) .GT. 1 .0) 

X(:,  :)  =  Temp(:,  :)**(1.0/aa) 
Y(:,  :)=X(:,  :)+  Temp2( : , : )**b 
end  where 
goto  100 
end  if 

C  print* , ’retry  =  ’,  retry 

101  call  cmf _random(Temp(l :N, 1 :N2)) 
if  (minval(Temp)  <=  0.0)  then 

C  print*, ’bad’ 

goto  101 
end  if 

102  call  cmf .random(Temp2(l :N,  1:N2)) 
if  (minval(Temp2)<=  0.0)  then 

C  print*, ’bad’ 

goto  102 
end  if 

RandGammal  *  X*(-log(Temp*Temp2)) 
end 


C  File  Random. h 

interface 

function  myRand(N,  N2,  PDF) 
integer  N,  N2 
character* 10  PDF 
real  myRand(N,N2) 
real  Temp(N,N2) 

CMF$  LAYOUT  myRauidC : serial ,  :news) ,  Temp( : serial ,  :news) 
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end  interface 
interface 

integer  function  whichPDF(PDF) 
character* 10  PDF 
end  interface 

interface 

function  RandUniform  (N,N2) 

integer  N,N2 

real  R2indUniform(N,N2) 

CMF$  LAYOUT  RandUniformC : serial,  rnews) 

end  interface 

interface 

function  RandGaussian  (M,K2) 
integer  N,N2 
real  RandGaussian(N,N2) 
real  Temp(M.M2) 

CMF$  LAYOUT  RandGaussian( : serial,  :news),  Temp( rserial,  :news) 

end  interface 
interface 

function  RandLaplacieui  (N,N2) 
integer  M,N2 

real  RandLaplacian(N,N2) 
real  Temp(N,N2) 

CMF$  LAYOUT  RandLaplacianC : serial,  inews) ,  Temp( :serial,  ;news) 

end  interface 
interface 

function  RandCauchy  (M,  M2) 
integer  N,N2 
real  RandCauchy (N,  N2) 
real  TempCN,  N2) 

CMF$  LAYOUT  RandCauchy (: serial,  mews),  Temp( rserial ,  rnews) 

end  interface 
interface 

function  RandGenGauss4  (M,M2) 
integer  N,N2 

real  RandGenGauss4(N,N2) 
real  Temp(N,N2) 

CMF$  LAYOUT  RandGenGauss4( rserial,  -.news),  Temp( rserial,  rnews) 

end  interface 

interface 

function  erf(NMAX,  N2MAX,  R,  x) 
real  erf 

integer  NMAX,  N2HAX 
real  R(NNAX,  N2MAX) 
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real  x 

CMF$  LAYOUT  R(: serial.  :news) 

end  interface 


C  File  LogLRT.h 

interface 

fmiction  LogLRT(NN,N2,  R,  a,  PDF,  s) 

Real  LogLRT(NN.N2) 

character* 10  PDF  !  which  pdf 

Integer  NN,  N2  !  vector  length,  number  of  channels 

Real  R(NN,  N2) ,  a,  s 

CMF$  LAYOUT  LogLRT( : serial,  :ne«s),  R(;serial,  :news) 

end  interface 
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